This document is licensed under a Creative Commons Attribution 4.0 International License.
The code is licensed under the MIT License.
The data used is licensed under a Open Data Commons Attribution License.
See the paper’s “Author Contributions” section for details on the contributors of data files.
Required libraries and runtime environment description.
This document is versioned in a public git repo: https://github.com/cgranell/apps-treatment-mental-disorders.
This R notebook contains the code to produce the final figures and tables of the paper.
The final number of surveyed papers is 158. For each paper, we have extracted 70 items. Each item is described in the methods section of the paper.
About 71.5% (N=113) of papers reported some kind of assessment, whereas 28.5% (N=45) reported no assessment at all. Looking closer at the type of assessment: 13.9% (N=22) focus on the effect of intervention on clinical symptomology; 32.3% (N=51) report a mix of effect and usability / UX assessment; 25.3% (N=40) focus solely on usability / UX.
It combines two plots. The top one is a stacked bar chart over years, and the value of each group (assessment type) is in absolute counts. The bottom one is a proportional stacked area chart in which the sum of each year is always equal to hundred, and the value of each group (assessment type) is in percentages.
NOTE: In response to the comment G.26 ‘Characteristics of included studies’.
NOTE: In response to the comment G.18 “The authors may consider presenting the data in the ‘Evolution of research and assessment’ paragraph using a table instead”. The following table simplifies the last paragragh of the section ‘Evolution of research and assessment’.
| Assessment type | N (%) | RCT - N (%) | PILOT RCT - N (%) |
|---|---|---|---|
| NO ASSESSMENT | 45 (28.5%) | 0 (0%) | 0 (0%) |
| USABILITY/UX | 40 (25.3%) | 2 (1.3%) | 0 (0%) |
| EFFECT AND USABILITY/UX | 51 (32.3%) | 14 (8.9%) | 5 (3.2%) |
| EFFECT | 22 (13.9%) | 9 (5.7%) | 2 (1.3%) |
The main plot is a stacked barchart colored by assessment type along with a lollipop chart (white circle at the end of each bar) that shows the total percentatge of each mental disorder. Note that we draw two main colors in the legend: purple represents no assessment at all, the range of greens denotes distinct types of assessment. A pie chart is embedded to show the proportional distribution of types of assessment over all mental disorders.
To complement the previous figure, some percentages are discussed in the section ‘Relation between assessment types and mental disorder’ which are included in the following table (that’s not included in the paper).
| Mental Disorder | Type of assessment | Studies (N) | Studies (%) |
|---|---|---|---|
| Depressive disorders | EFFECT | 6 | 19.4% |
| EFFECT AND USABILITY/UX | 12 | 38.7% | |
| NO ASSESSMENT | 9 | 29% | |
| USABILITY/UX | 4 | 12.9% | |
| Various disorders | EFFECT | 2 | 9.5% |
| EFFECT AND USABILITY/UX | 7 | 33.3% | |
| NO ASSESSMENT | 4 | 19% | |
| USABILITY/UX | 8 | 38.1% | |
| Anxiety disorders | EFFECT | 5 | 27.8% |
| EFFECT AND USABILITY/UX | 4 | 22.2% | |
| NO ASSESSMENT | 8 | 44.4% | |
| USABILITY/UX | 1 | 5.6% | |
| Substance-related and addictive disorders | EFFECT | 2 | 12.5% |
| EFFECT AND USABILITY/UX | 5 | 31.2% | |
| NO ASSESSMENT | 8 | 50% | |
| USABILITY/UX | 1 | 6.2% | |
| Schizophrenia spectrum and other psychotic disorders | EFFECT AND USABILITY/UX | 4 | 26.7% |
| NO ASSESSMENT | 5 | 33.3% | |
| USABILITY/UX | 6 | 40% | |
| Trauma and stressor-related disorders | EFFECT | 2 | 13.3% |
| EFFECT AND USABILITY/UX | 7 | 46.7% | |
| NO ASSESSMENT | 1 | 6.7% | |
| USABILITY/UX | 5 | 33.3% | |
| Suicidal behavior disorder/nonsuicidal self-injury | EFFECT | 1 | 9.1% |
| EFFECT AND USABILITY/UX | 5 | 45.5% | |
| NO ASSESSMENT | 3 | 27.3% | |
| USABILITY/UX | 2 | 18.2% | |
| Comorbid disorders | EFFECT | 1 | 14.3% |
| NO ASSESSMENT | 2 | 28.6% | |
| USABILITY/UX | 4 | 57.1% | |
| Bipolar and related disorders | EFFECT | 1 | 16.7% |
| EFFECT AND USABILITY/UX | 2 | 33.3% | |
| USABILITY/UX | 3 | 50% | |
| Obsessive-Compulsive and related disorders | EFFECT | 1 | 20% |
| EFFECT AND USABILITY/UX | 3 | 60% | |
| USABILITY/UX | 1 | 20% | |
| Neurodevelopmental disorders | EFFECT AND USABILITY/UX | 1 | 25% |
| NO ASSESSMENT | 1 | 25% | |
| USABILITY/UX | 2 | 50% | |
| Feeding and eating disorders | NO ASSESSMENT | 1 | 25% |
| USABILITY/UX | 3 | 75% | |
| Sleep-wake disorders | EFFECT | 1 | 33.3% |
| NO ASSESSMENT | 2 | 66.7% | |
| Personality disorders | EFFECT AND USABILITY/UX | 1 | 100% |
| Major and Mild Neurocognitive Disorders | NO ASSESSMENT | 1 | 100% |
NOTE: In respond to the comment F.2 ‘Breakdown of treatment modalities’ to show psychological framework per study (ALL)
NOTE: In respond to the comment A.14 ‘Percentage of apps having been studied in multiple articles’
Out of N=158, there are 138 studies with app names. However, there are 108 unique apps. The unique apps follow a long tail distribution, since 89 appear only once, while 19 apps appear in two or more studies, being two studies the most frequent case (14) and apps in 3 or more studies occurs in only 5 cases. In summary, the percentage of apps having been studied in multiple articles is 108 / 19 (13.8%).
Note also that 3 apps are classified in two categories of mental disorders.
A line chart to show the number of papers per mental disorder and year.
A tabular, compact distribution of apps per mental disorder, grouping the references tha point the same app together. The number(s) in brackets next to the app name is the reference(s) in which the app is mentioned. In the table below, these references are internal identifiers instead.
| Mental Disorder | References by app |
|---|---|
| Major and Mild Neurocognitive Disorders | Rico (452) |
| Personality disorders | EMOTEO (106) |
| Sleep-wake disorders | Sleepcare (181), SleepIO (094;103) |
| Feeding and eating disorders | Jorvie (265), Student Bodies–Eating Disorders (SB-ED) (007), Recovery Record (092;460) |
| Neurodevelopmental disorders | iCanLearn (125), LifePal (142), My MFG (304), TimeOut (454) |
| Obsessive-Compulsive and related disorders | Geo-Feedback App (262), GGOC (473), Live OCD Free (044), Mayo Clinic Anxiety Coach (056), RAW HAND (448) |
| Bipolar and related disorders | MyT (042), PRISM (048), SIMPLe (258;226;138;459) |
| Comorbid disorders | CASA-CHESS (297), Enuresis Trainer (290), Learn To Quit (300), SMI-CM (232), Stay Quit Coach (463), Stop-Cannabis (097), NA (086) |
| Suicidal behavior disorder/nonsuicidal self-injury | BackUp; mEMA (453), BeyondNow (409), BRITE (470), iBobbly (124), Virtual Hope Box (VHB) (026;443), NA (275;012), Blue Ice (245;022;030) |
| Trauma and stressor-related disorders | RELAX (301), NA (302), Life Armor; PE Coach; Positive Activity Jackpot; Eventful; Tactical Breather; Virtual Hope Box; Daily Yoga; Simply Yoga (121;037), PE Coach (062;061;069;049), PTSD Coach (160;118;306;015;088;450;478) |
| Schizophrenia spectrum and other psychotic disorders | iCOPE (090), MindFrame (471), movisenseXS (420), RealLife Exp (419), SlowMo (260), TechCare (273), Temstem (418), Actissist (305;440), FOCUS (089;120), Heal Your Mind (087;406), PRIME (117;408) |
| Substance-related and addictive disorders | Drink Less (468), Fit&Sobber (411), Mind the Moment (171), S-Health (437), SEVA (127), SmartQuit (217), Smoke Mind (065), Social-Local-Mobile (430), A-CHESS (013;307), CET Application (014;021), Kick.it (402;413), Smart-T (038;039) |
| Anxiety disorders | Agoraphobia Free; Stress Free (291), Ångesthjälpen (122), AnxietyCoach (192), CBT Assistant (057), Challenger (464), Lantern (025), PsychAssist (286), Public Speech Trainer (PST) (050), SmartCAT (425), GET.ON PAPP (091;105), NA (123;308;027;285;163;457;472) |
| Various disorders | ACT Daily APP (110), FOCUS (414), Headspace (177), iBobbly (029), iCare-stress (433), IntelliCare Suite (131), MoodMission (427), MoodTrainer (185), myCompass (126), PeerTECH App (119), Pocket Skills (436), Sinasprite (431), SmartCAT (282), SPIRIT (140), The Moment (281), TODAY! (294), Wellframe (034), WellWave (477), NA (475), WorkingWell (220;434) |
| Depressive disorders | 7Cups (082), Be Good to Yourself (466), BlueWatch (401), Dcombat (235), Get Happy Program (058), HeadGear (438), iCare Prevent (455), MedLink (152), Mobile Sensing and Support (183), MoodHacker (187), Moodivate (188), MyGamePlan (203), PRIME-D (279), Push-D (407), SocioEmpathy (421), SPSRS (115), SuperBetter (231), The Sound Advice (428), Thought Challenger (114), TODAC TODAC (020), Kokoro-App (035;263;063), NA (051;023;266;071;303;077;432;435) |
NOTE: In response to the comment G.26 ‘Characteristics of included studies’. It adds psychological framework data per study to the big table. In production (manual formating), the table can get more compact, all “NO” can be deleted (empty cell instead), and “YES” can be substituted by coloring the cell depending on the psychological approach, as in the example I provided.
| Mental Disorder | App name | Ref | CBT | WAV | TRA | POS |
|---|---|---|---|---|---|---|
| Major and Mild Neurocognitive Disorders | Rico | 452 | NO | NO | NO | NO |
| Personality disorders | EMOTEO | 106 | NO | YES | NO | NO |
| Sleep-wake disorders | Sleepcare | 181 | YES | NO | NO | NO |
| SleepIO | 094 | YES | YES | NO | YES | |
| 103 | YES | NO | NO | NO | ||
| Feeding and eating disorders | Jorvie | 265 | NO | YES | NO | NO |
| Recovery Record | 092 | YES | YES | NO | NO | |
| 460 | YES | NO | NO | NO | ||
| Student Bodies–Eating Disorders (SB-ED) | 007 | YES | NO | NO | NO | |
| Neurodevelopmental disorders | iCanLearn | 125 | YES | NO | NO | NO |
| LifePal | 142 | NO | NO | NO | NO | |
| My MFG | 304 | NO | NO | NO | NO | |
| TimeOut | 454 | YES | NO | NO | NO | |
| Obsessive-Compulsive and related disorders | Geo-Feedback App | 262 | YES | NO | NO | NO |
| GGOC | 473 | YES | NO | NO | NO | |
| Live OCD Free | 044 | YES | NO | NO | NO | |
| Mayo Clinic Anxiety Coach | 056 | YES | YES | NO | NO | |
| RAW HAND | 448 | YES | NO | NO | NO | |
| Bipolar and related disorders | MyT | 042 | YES | NO | NO | NO |
| PRISM | 048 | YES | NO | NO | NO | |
| SIMPLe | 138 | NO | NO | NO | NO | |
| 226 | NO | NO | NO | NO | ||
| 258 | NO | NO | NO | NO | ||
| 459 | NO | NO | NO | NO | ||
| Comorbid disorders | CASA-CHESS | 297 | YES | NO | NO | NO |
| Enuresis Trainer | 290 | YES | NO | NO | NO | |
| Learn To Quit | 300 | YES | YES | NO | NO | |
| SMI-CM | 232 | NO | YES | NO | NO | |
| Stay Quit Coach | 463 | YES | NO | NO | NO | |
| Stop-Cannabis | 097 | YES | NO | NO | NO | |
| NA | 086 | NO | YES | NO | NO | |
| Suicidal behavior disorder/nonsuicidal self-injury | BackUp; mEMA | 453 | YES | NO | NO | NO |
| BeyondNow | 409 | NO | NO | NO | NO | |
| Blue Ice | 022 | YES | YES | NO | NO | |
| 030 | YES | YES | NO | NO | ||
| 245 | YES | YES | NO | NO | ||
| BRITE | 470 | YES | YES | NO | YES | |
| iBobbly | 124 | NO | YES | NO | YES | |
| Virtual Hope Box (VHB) | 026 | YES | NO | NO | YES | |
| 443 | YES | NO | NO | NO | ||
| NA | 012 | YES | NO | NO | YES | |
| NA | 275 | NO | YES | NO | NO | |
| Trauma and stressor-related disorders | Life Armor; PE Coach; Positive Activity Jackpot; Eventful; Tactical Breather; Virtual Hope Box; Daily Yoga; Simply Yoga | 037 | YES | NO | NO | NO |
| 121 | YES | NO | NO | NO | ||
| PE Coach | 049 | YES | NO | NO | NO | |
| 061 | YES | YES | NO | NO | ||
| 062 | YES | YES | NO | NO | ||
| 069 | YES | NO | NO | NO | ||
| PTSD Coach | 015 | YES | NO | NO | NO | |
| 088 | YES | NO | NO | NO | ||
| 118 | YES | NO | NO | NO | ||
| 160 | YES | NO | NO | NO | ||
| 306 | YES | NO | NO | NO | ||
| 450 | YES | NO | NO | NO | ||
| 478 | YES | NO | NO | NO | ||
| RELAX | 301 | YES | NO | NO | NO | |
| NA | 302 | YES | NO | NO | NO | |
| Schizophrenia spectrum and other psychotic disorders | Actissist | 305 | YES | NO | NO | NO |
| 440 | YES | YES | NO | NO | ||
| FOCUS | 089 | YES | NO | NO | NO | |
| 120 | YES | NO | NO | NO | ||
| Heal Your Mind | 087 | YES | NO | NO | NO | |
| 406 | YES | NO | NO | NO | ||
| iCOPE | 090 | YES | NO | NO | NO | |
| MindFrame | 471 | NO | NO | NO | NO | |
| movisenseXS | 420 | YES | NO | NO | NO | |
| PRIME | 117 | YES | YES | NO | NO | |
| 408 | NO | NO | NO | NO | ||
| RealLife Exp | 419 | YES | NO | NO | NO | |
| SlowMo | 260 | NO | NO | NO | NO | |
| TechCare | 273 | YES | NO | NO | NO | |
| Temstem | 418 | NO | NO | NO | NO | |
| Substance-related and addictive disorders | A-CHESS | 013 | YES | NO | NO | NO |
| 307 | NO | NO | NO | NO | ||
| CET Application | 014 | YES | NO | NO | NO | |
| 021 | YES | NO | NO | NO | ||
| Drink Less | 468 | YES | NO | NO | NO | |
| Fit&Sobber | 411 | NO | NO | NO | NO | |
| Kick.it | 402 | YES | NO | NO | NO | |
| 413 | YES | NO | NO | NO | ||
| Mind the Moment | 171 | YES | NO | NO | NO | |
| S-Health | 437 | YES | NO | NO | NO | |
| SEVA | 127 | YES | NO | NO | NO | |
| Smart-T | 038 | YES | NO | NO | NO | |
| 039 | YES | NO | NO | NO | ||
| SmartQuit | 217 | NO | YES | NO | NO | |
| Smoke Mind | 065 | YES | NO | NO | NO | |
| Social-Local-Mobile | 430 | YES | NO | NO | NO | |
| Anxiety disorders | Agoraphobia Free; Stress Free | 291 | YES | NO | YES | NO |
| Ångesthjälpen | 122 | NO | YES | NO | NO | |
| AnxietyCoach | 192 | YES | NO | NO | NO | |
| CBT Assistant | 057 | YES | YES | NO | NO | |
| Challenger | 464 | YES | NO | NO | NO | |
| GET.ON PAPP | 091 | YES | NO | NO | NO | |
| 105 | YES | NO | NO | NO | ||
| Lantern | 025 | YES | NO | NO | NO | |
| PsychAssist | 286 | NO | NO | YES | NO | |
| Public Speech Trainer (PST) | 050 | NO | NO | NO | NO | |
| SmartCAT | 425 | YES | NO | NO | NO | |
| NA | 027 | YES | NO | NO | YES | |
| NA | 123 | YES | YES | YES | NO | |
| NA | 163 | YES | NO | NO | NO | |
| NA | 285 | YES | NO | NO | NO | |
| NA | 308 | YES | NO | NO | NO | |
| NA | 457 | YES | NO | NO | NO | |
| NA | 472 | YES | NO | NO | NO | |
| Various disorders | ACT Daily APP | 110 | NO | YES | NO | NO |
| FOCUS | 414 | YES | NO | NO | NO | |
| Headspace | 177 | NO | YES | NO | NO | |
| iBobbly | 029 | YES | YES | NO | NO | |
| iCare-stress | 433 | YES | NO | NO | NO | |
| IntelliCare Suite | 131 | YES | NO | NO | YES | |
| MoodMission | 427 | YES | YES | NO | NO | |
| MoodTrainer | 185 | YES | NO | NO | NO | |
| myCompass | 126 | YES | YES | NO | YES | |
| PeerTECH App | 119 | NO | NO | NO | NO | |
| Pocket Skills | 436 | NO | YES | NO | NO | |
| Sinasprite | 431 | YES | YES | NO | NO | |
| SmartCAT | 282 | YES | NO | NO | NO | |
| SPIRIT | 140 | YES | NO | NO | NO | |
| The Moment | 281 | YES | YES | NO | YES | |
| TODAY! | 294 | YES | NO | YES | NO | |
| Wellframe | 034 | YES | NO | NO | NO | |
| WellWave | 477 | NO | NO | NO | NO | |
| WorkingWell | 220 | NO | NO | NO | NO | |
| 434 | YES | NO | NO | NO | ||
| NA | 475 | YES | NO | NO | NO | |
| Depressive disorders | 7Cups | 082 | NO | YES | NO | NO |
| Be Good to Yourself | 466 | YES | YES | NO | NO | |
| BlueWatch | 401 | YES | NO | NO | NO | |
| Dcombat | 235 | YES | NO | NO | NO | |
| Get Happy Program | 058 | YES | YES | NO | NO | |
| HeadGear | 438 | YES | YES | NO | NO | |
| iCare Prevent | 455 | YES | NO | NO | NO | |
| Kokoro-App | 035 | YES | NO | NO | NO | |
| 063 | YES | NO | NO | NO | ||
| 263 | YES | NO | NO | NO | ||
| MedLink | 152 | YES | NO | NO | YES | |
| Mobile Sensing and Support | 183 | YES | NO | NO | NO | |
| MoodHacker | 187 | YES | YES | NO | YES | |
| Moodivate | 188 | NO | YES | NO | NO | |
| MyGamePlan | 203 | NO | NO | NO | NO | |
| PRIME-D | 279 | YES | NO | NO | NO | |
| Push-D | 407 | YES | NO | NO | YES | |
| SocioEmpathy | 421 | NO | NO | NO | NO | |
| SPSRS | 115 | YES | NO | NO | NO | |
| SuperBetter | 231 | YES | YES | NO | YES | |
| The Sound Advice | 428 | YES | NO | NO | NO | |
| Thought Challenger | 114 | YES | NO | NO | NO | |
| TODAC TODAC | 020 | YES | NO | NO | NO | |
| NA | 023 | YES | YES | NO | NO | |
| NA | 051 | YES | YES | NO | NO | |
| NA | 071 | YES | NO | NO | NO | |
| NA | 077 | YES | NO | NO | NO | |
| NA | 266 | YES | YES | NO | NO | |
| NA | 303 | YES | NO | NO | NO | |
| NA | 432 | YES | NO | NO | NO | |
| NA | 435 | YES | NO | NO | NO |